On Multiplicative Entropy and Information gain in Large Data Sets

Information theory is one of the widely used branches of applied probability theory. When probability is used to describe the state of a system implies that the state has some uncertainty. Some probability distributions indicate more uncertainty than others as they are not created equal. We can come up with some mathematical entity which returns a measure of uncertainty after taking a probability distribution as input. It has been observed that the mutual information between two variables is the reduction in uncertainty of one variable due to information of other. In this paper a new approach is taken to look into the multiplicative nature of entropy and the conditional entropy. Then on the basis of this information gain is calculated using a large data set. The data set considered, are the scanned OMR application forms of the candidates applying in engineering courses of a University. Simulation has been done using such data and information gain is calculated using some predefined parameters.